Fake News Detection Using Transformer and Ensemble Learning Models

On social media, misinformation can spread quickly, posing serious problems. Understanding the content and sensitive nature of fake news and misinformation is critical to prevent the damage caused by them. To this end, the characteristics of information must first be discerned. In this paper, we propose a transformer-based hybrid ensemble model to detect misinformation on the Internet. First, false and true news on Covid-19 were analyzed, and various text classification tasks were performed to understand their content. The results were utilized in the proposed hybrid ensemble learning model. Our analysis revealed promising results, establishing the capability of the proposed system to detect misinformation on social media. The final model exhibited an excellent F1 score (0.98) and accuracy (0.97). The AUC (Area Under The Curve) score was also high at 0.98, and the ROC (Receiver Operating Characteristics) curve revealed that the true-positive rate of the data was close to one in this model. Thus, the proposed hybrid model was demonstrated to be successful in recognizing false information online. © 2022 IEEE.

Analyzing COVID-Related Social Discourse on Twitter using Emotion, Sentiment, Political Bias, Stance, Veracity and Conspiracy Theories

Online misinformation has become a major concern in recent years, and it has been further emphasized during the COVID-19 pandemic. Social media platforms, such as Twitter, can be serious vectors of misinformation online. In order to better understand the spread of these fake-news, lies, deceptions, and rumours, we analyze the correlations between the following textual features in tweets: emotion, sentiment, political bias, stance, veracity and conspiracy theories. We train several transformer-based classifiers from multiple datasets to detect these textual features and identify potential correlations using conditional distributions of the labels. Our results show that the online discourse regarding some topics, such as COVID-19 regulations or conspiracy theories, is highly controversial and reflects the actual U.S. political landscape. © 2023 ACM.

A Classification Approach to Detect Public Sentiments towards COVID-19 Vaccines

In this paper, team University of Botswana Computer Science (UBCS) investigate the opinions of Twitter users towards vaccine uptake. In particular, we build three different text classifiers to detect people's opinions and classify them as provax-for opinions that are for vaccination, antivax for opinions against vaccination and neutral-for opinions that are neither for or against vaccination. Two different datasets obtained from Twitter, 1 by Cotfas and the other by Fire2022 Organizing team were merged to and used for this study. The dataset contained 4392 tweets. Our first classifier was based on the basic BERT model and the other 2 were machine learning models, Random Forest and Multinomial Naive Bayes models. Naive Bayes classifier outperformed other classifiers with a macro-F1 score of 0.319. © 2022 Copyright for this paper by its authors.

CTC: COVID-19 Tweet Classification using CT-BERT

CTC is my submitted work to the Information Retrieval from Microblogs during Disasters (IRMiDis) Track at the Forum for Information Retrieval Evaluation (FIRE) 2022. Coronavirus disease (COVID-19) is an infectious disease caused by the SARS-CoV-2 virus. Most people infected with the virus experience a mild to moderate respiratory illness and recover without requiring special treatment. However, some become seriously ill and require medical attention. Vaccines against coronavirus and prompt reporting of symptoms saved many lives during the pandemic. The analysis of COVID-19-related tweets can provide valuable insights regarding the stance of people toward the new vaccine. It can also help the authorities to plan their strategies based on people's opinions about the vaccine and ensure the effectiveness of vaccination campaigns. Tweets describing symptoms can also aid in identifying high-alert zones and determining quarantine regulations. The IRMiDis track focuses on these COVID-19-related tweets that flooded Twitter. I developed an effective classifier for both Tasks 1 and 2. The evaluation score of my submitted run is reported in terms of accuracy and macro-F1 score. I achieved an accuracy of 0.770, a macro-F1 score of 0.773 in Task 1, and an accuracy of 0.820, a macro-F1 score of 0.746 in Task 2. I enjoyed the first rank among other submissions in both the tasks. © 2022 Copyright for this paper by its authors.

Analysing sentiment change detection of Covid-19 tweets.

The Covid-19 pandemic made a significant impact on society, including the widespread implementation of lockdowns to prevent the spread of the virus. This measure led to a decrease in face-to-face social interactions and, as an equivalent, an increase in the use of social media platforms, such as Twitter. As part of Industry 4.0, sentiment analysis can be exploited to study public attitudes toward future pandemics and sociopolitical situations in general. This work presents an analysis framework by applying a combination of natural language processing techniques and machine learning algorithms to classify the sentiment of each tweet as positive, or negative. Through extensive experimentation, we expose the ideal model for this task and, subsequently, utilize sentiment predictions to perform time series analysis over the course of the pandemic. In addition, a change point detection algorithm was applied in order to identify the turning points in public attitudes toward the pandemic, which were validated by cross-referencing the news report at that particular period of time. Finally, we study the relationship between sentiment trends on social media and, news coverage of the pandemic, providing insights into the public's perception of the pandemic and its influence on the news.

Classification of Highly Divergent Viruses from DNA/RNA Sequence Using Transformer-Based Models.

Viruses infect millions of people worldwide each year, and some can lead to cancer or increase the risk of cancer. As viruses have highly mutable genomes, new viruses may emerge in the future, such as COVID-19 and influenza. Traditional virology relies on predefined rules to identify viruses, but new viruses may be completely or partially divergent from the reference genome, rendering statistical methods and similarity calculations insufficient for all genome sequences. Identifying DNA/RNA-based viral sequences is a crucial step in differentiating different types of lethal pathogens, including their variants and strains. While various tools in bioinformatics can align them, expert biologists are required to interpret the results. Computational virology is a scientific field that studies viruses, their origins, and drug discovery, where machine learning plays a crucial role in extracting domain- and task-specific features to tackle this challenge. This paper proposes a genome analysis system that uses advanced deep learning to identify dozens of viruses. The system uses nucleotide sequences from the NCBI GenBank database and a BERT tokenizer to extract features from the sequences by breaking them down into tokens. We also generated synthetic data for viruses with small sample sizes. The proposed system has two components: a scratch BERT architecture specifically designed for DNA analysis, which is used to learn the next codons unsupervised, and a classifier that identifies important features and understands the relationship between genotype and phenotype. Our system achieved an accuracy of 97.69% in identifying viral sequences.

Analysis of Learners' Sentiments on MOOC Forums using Natural Language Processing Techniques

MOOCs have gained a lot of popularity for past few years. Especially after the outbreak of Coronavirus, everyone is trying to gain some knowledge and skill while being at the comfort of home and making themselves safe. Due to sudden increase in the number of participants on MOOCs there is a need for an automated system to be able to assess the reviews and feedbacks given by the learners and find the sentiments behind their statements. This analysis will help trainers identify their shortcoming and make their courses even better. For sentiments analysis, several approaches may be used. This research aims to provide a system which will perform sentiments analysis on the novel dataset and show the comparison of lexicon-based vs transformer-based sentiment analysis models. For lexicon based, VADER was chosen and for transformer-based, state-of-The-Art BERT was chosen. BERT was found to be exceptionally good with an accuracy of 84% and F1-score of 0.64. © 2022 IEEE.

COVID QA Network: A Specific Case of Biomedical Question Answering

COVID-19 crisis has led to an outburst of information that needs to be organized, validated, and made available to the seekers. Despite the rapid growth and success of BERT models in the last 3 years, COVID QA is a difficult task due to the lack of applicable datasets and a relevant language representation. Therefore, this study proposes a transformer-based Question Answering (QA) model for COVID-19 questions from the biomedical domain. Further, explored several datasets, and models required for question type prediction, no-Answer prediction, and answer extraction and transfer learning strategies. It has been demonstrated that the exact match score can be significantly improved with limited amounts of training data from the biomedical domain. Finally, the findings of the study have been summarized as Factoid QA Finetuning Framework (FQFF), which can provide initial direction for domain-specific QA tasks with a limited amount of data. © 2023 IEEE.

Arabic fake news detection in social media Based on AraBERT

Social media has become a source of information for many people because of its freedom of use. As a result, fake news spread quickly and easily, regardless of its credibility, especially over the past decade. The vast amount of information being shared has fraudulent practices that negatively affect readers' cognitive abilities and mental health. In this study, we aim to introduce a new Arabic COVID-19 dataset for fake news related to COVID-19 from Twitter and Facebook. Afterward, we applied two pre-Trained models of classification AraBERT and BERT base Arabic. As a result, AraBERT models obtained better accuracy than BERT base Arabic in two datasets. © 2022 IEEE.

Investigating Canadian Public Attitudes Toward COVID-19 Vaccine Mandates with a Nested Analysis Framework.

BACKGROUND: Social media is an important medium for studying public attitudes toward COVID-19 vaccine mandates in Canada, and Reddit network communities are a good source for this. METHODS: This study applied a "nested analysis" framework. We collected 20378 Reddit comments via the Pushshift API and developed a BERT-based binary classification model to screen for relevance to COVID-19 vaccine mandates. We then used a Guided Latent Dirichlet Allocation (LDA) model on relevant comments to extract key topics and assign each comment to its most relevant topic. RESULTS: There were 3179 (15.6%) relevant and 17199 (84.4%) irrelevant comments. Our BERT-based model achieved 91% accuracy trained with 300 Reddit comments after 60 epochs. The Guided LDA model had an optimal coherence score of 0.471 with four topics: travel, government, certification, and institutions. Human evaluation of the Guided LDA model showed an 83% accuracy in assigning samples to their topic groups. CONCLUSION: We develop a screening tool for filtering and analyzing Reddit comments on COVID-19 vaccine mandates through topic modelling. Future research could develop more effective seed word-choosing and evaluation methods to reduce the need for human judgment.

Vaccine sentiment analysis using BERT + NBSVM and geo-spatial approaches.

Since the spread of the coronavirus flu in 2019 (hereafter referred to as COVID-19), millions of people worldwide have been affected by the pandemic, which has significantly impacted our habits in various ways. In order to eradicate the disease, a great help came from unprecedentedly fast vaccines development along with strict preventive measures adoption like lockdown. Thus, world wide provisioning of vaccines was crucial in order to achieve the maximum immunization of population. However, the fast development of vaccines, driven by the urge of limiting the pandemic caused skeptical reactions by a vast amount of population. More specifically, the people's hesitancy in getting vaccinated was an additional obstacle in fighting COVID-19. To ameliorate this scenario, it is important to understand people's sentiments about vaccines in order to take proper actions to better inform the population. As a matter of fact, people continuously update their feelings and sentiments on social media, thus a proper analysis of those opinions is an important challenge for providing proper information to avoid misinformation. More in detail, sentiment analysis (Wankhade et al. in Artif Intell Rev 55(7):5731-5780, 2022. 10.1007/s10462-022-10144-1) is a powerful technique in natural language processing that enables the identification and classification of people feelings (mainly) in text data. It involves the use of machine learning algorithms and other computational techniques to analyze large volumes of text and determine whether they express positive, negative or neutral sentiment. Sentiment analysis is widely used in industries such as marketing, customer service, and healthcare, among others, to gain actionable insights from customer feedback, social media posts, and other forms of unstructured textual data. In this paper, Sentiment Analysis will be used to elaborate on people reaction to COVID-19 vaccines in order to provide useful insights to improve the correct understanding of their correct usage and possible advantages. In this paper, a framework that leverages artificial intelligence (AI) methods is proposed for classifying tweets based on their polarity values. We analyzed Twitter data related to COVID-19 vaccines after the most appropriate pre-processing on them. More specifically, we identified the word-cloud of negative, positive, and neutral words using an artificial intelligence tool to determine the sentiment of tweets. After this pre-processing step, we performed classification using the BERT + NBSVM model to classify people's sentiments about vaccines. The reason for choosing to combine bidirectional encoder representations from transformers (BERT) and Naive Bayes and support vector machine (NBSVM ) can be understood by considering the limitation of BERT-based approaches, which only leverage encoder layers, resulting in lower performance on short texts like the ones used in our analysis. Such a limitation can be ameliorated by using Naive Bayes and Support Vector Machine approaches that are able to achieve higher performance in short text sentiment analysis. Thus, we took advantage of both BERT features and NBSVM features to define a flexible framework for our sentiment analysis goal related to vaccine sentiment identification. Moreover, we enrich our results with spatial analysis of the data by using geo-coding, visualization, and spatial correlation analysis to suggest the most suitable vaccination centers to users based on the sentiment analysis outcomes. In principle, we do not need to implement a distributed architecture to run our experiments as the available public data are not massive. However, we discuss a high-performance architecture that will be used if the collected data scales up dramatically. We compared our approach with the state-of-art methods by comparing most widely used metrics like Accuracy, Precision, Recall and F-measure. The proposed BERT + NBSVM outperformed alternative models by achieving 73% accuracy, 71% precision, 88% recall and 73% F-measure for classification of positive sentiments while 73% accuracy, 71% precision, 74% recall and 73% F-measure for classification of negative sentiments respectively. These promising results will be properly discussed in next sections. The use of artificial intelligence methods and social media analysis can lead to a better understanding of people's reactions and opinions about any trending topic. However, in the case of health-related topics like COVID-19 vaccines, proper sentiment identification could be crucial for implementing public health policies. More in detail, the availability of useful findings on user opinions about vaccines can help policymakers design proper strategies and implement ad-hoc vaccination protocols according to people's feelings, in order to provide better public service. To this end, we leveraged geospatial information to support effective recommendations for vaccination centers.

Sentiment analysis and counselling for COVID-19 pandemic based on social media

As COVID-19 emerged and prolonged, various changes have occurred in our lives. For example, as restrictions on daily life are lengthening, the number of people complaining of depression is increasing. In this paper, we conduct a sentiment analysis by modelling public emotions and issues through social media. Text data written on Twitter is collected by dividing it into the early and late stages of COVID-19, and emotional analysis is performed to reclassify it into positive and negative tweets. Therefore, subject modelling is performed with a total of four datasets to review the results and evaluate the modelling results. Furthermore, topic modelling results are visualised using dimensional reduction, and public opinions on COVID-19 are intuitively confirmed by generating representative words consisting of each topic in the word cloud. Additionally, we implement a COVID-chatbot that provides a question-and-answer service on COVID-19 and verifies the performance in our experiments.

N-Gram-Codon and Recurrent Neural Network (RNN) to Update Pfizer-BioNTech mRNA Vaccine

In the fight against SARS-CoV-2, Pfizer BioNTech based on synthetic messenger RNA (mRNA) proved to be quicker and more effective even with a small dose of micrograms per injection. Unfortunately, such a vaccine requires very low temperatures to prevent degradation of mRNA. In this paper, the authors have developed three new models of recurrent neural network (1-simple LSTM 2-BDLSTM 3-BERT) using n-gram-codon technique for the codification of mRNA. The primary aim is to analyse the mRNA sequence and predict the stability/reactivity rates at various codon positions. The results of the predictions will be presented in the form of recommendations to support laboratories in updating Pfizer's BioNTech vaccine. The obtained results were validated by the Stanford OpenVaccine dataset and the evaluation measures recall, precision, f1-score, accuracy, and loss.

Text Analysis of COVID-19 Tweets

During the COVID-19 pandemic many countries were forced to implement lockdowns to prevent further spread of the SARS-CoV-2, prohibiting people from face-to-face social interactions. This unprecedented circumstance led to an increase in traffic on social media platforms, one of the most popular of which is Twitter, with a diverse spectrum of users from around the world. This quality, along with the ability to use its API for research purposes, makes it a valuable resource for data collection and analysis. In this paper we aim to present the sentiments towards the COVID-19 pandemic and vaccines as it was imprinted through the users' tweets when the events were actually still in motion. For our research, we gathered the related data from Twitter and characterized the gathered tweets in two classes, positive and negative;using the BERT model, with an accuracy of 99%. Finally, we performed various time series analyses based on people's sentiment with reference to the pandemic period of 2021, the four major vaccine's companies as well as on the vaccine's technology.

Transfer Learning for Closed Domain Question Answering in COVID-19

COVID-19 has been a popular issue around 2019 until today. Recently, there has been a lot of research being conducted to utilize a big amount of data discussing about COVID-19. In this work, we conduct a closed domain question answering (CDQA) task in COVID-19 using transfer learning technique. The transfer learning technique is adopted because a large benchmark for question answering about COVID-19 is still unavailable. Therefore, rich knowledge learned from a large benchmark of open domain QA are utilized using transfer learning to improve the performance of our CDQA system. We use retriever-reader framework for our CDQA system, and propose to use Sequential Dependence Model (SDM) as our retriever component to enhance the effectiveness of the system. Our result shows that the use of SDM retriever can improve the F-1 score of the state-of-the-art baseline CDQA system using BM25 and TF-IDF+cosine similarity retriever by 3,26% and 32,62%, respectively. The optimal parameter settings for our CDQA system are found to be as follows: using 20 top-ranked documents as the retriever's output, five sentences as the passage length, and BERT-Large-Uncased model as the reader. In this optimal parameter setting, SDM retriever can improve the F-1 score of the state-of-the-art baseline CDQA system using BM25 by 5,06 % and TF-IDF+cosine similarity retriever by 24,94 %. Our last experiment then confirms the merit of using transfer learning, since our best-performing model (double fine-tune SQuAD and COVID-QA) is shown to gain eight times higher accuracy than the baseline method without using transfer learning. Further fine-tuning the transfer learning model using closed domain dataset (COVID-QA) can increase the accuracy of the transfer learning model that only fine-tuning with SQuAD by 27, 26%.

Aspect-Based Sentiment Analysis with Semi-Supervised Approach on Taiwan Social Distancing App User Reviews

Sentiment analysis has a critical role to reveal an opinion in a text-based form. Therefore, we exploit this analysis to discover the sentiment polarity of Taiwan Social Distancing mobile application. This paper proposes a semi-supervised scheme for annotating this mobile application's reviews. The semi-supervised scheme utilized a combination of numeric rating and lexicon-based sentiment. In addition, we also perform the sentiment analysis on an aspect-based level. Based on the experiment, we decide to select three aspects to be analyzed. This paper also evaluates the proposed scheme by implementing bidirectional encoder representations from transformers (BERT) and multilayer perceptron (MLP) as the classification model using the sentiment label of the proposed scheme. The result shows that the annotation of the proposed scheme outperforms the data annotation using counterpart models. © 2023 IEEE.

Ontology-based semantic data interestingness using BERT models

The COVID-19 pandemic has generated massive data in the healthcare sector in recent years, encouraging researchers and scientists to uncover the underlying facts. Mining interesting patterns in the large COVID-19 corpora is very important and useful for the decision makers. This paper presents a novel approach for uncovering interesting insights in large datasets using ontologies and BERT models. The research proposes a framework for extracting semantically rich facts from data by incorporating domain knowledge into the data mining process through the use of ontologies. An improved Apriori algorithm is employed for mining semantic association rules, while the interestingness of the rules is evaluated using BERT models for semantic richness. The results of the proposed framework are compared with state-of-the-art methods and evaluated using a combination of domain expert evaluation and statistical significance testing. The study offers a promising solution for finding meaningful relationships and facts in large datasets, particularly in the healthcare sector. © 2023 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.

ADAPTATION OF DOMAIN-SPECIFIC TRANSFORMER MODELS WITH TEXT OVERSAMPLING FOR SENTIMENT ANALYSIS OF SOCIAL MEDIA POSTS ON COVID-19 VACCINE

Covid-19 has spread across the world, and several vaccines have been developed to counter its surge. To identify the correct sentiments that are associated with the vaccines from social media posts, we fine-tune various state-of-the-art pretrained transformer models on tweets that are associated with Covid-19 vaccines. Specifically, we use the recently introduced state-of-the-art RoBERTa, XLNet, and BERT pre-trained transformer models, and the domain-specific CT-BERT and BERTweet transformer models that have been pre-trained on Covid-19 tweets. We further explore the option of text augmentation by oversampling using the language model-based oversampling technique (LMOTE) to improve the accuracies of these models – specifically, for small sample data sets where there is an imbalanced class distribution among the positive, negative, and neutral sentiment classes. Our results summarize our findings on the suitability of text oversampling for imbalanced small-sample data sets that are used to fine-tune state-of-the-art pre-trained transformer models as well as the utility of domain-specific transformer models for the classification task. © 2023 Author(s). This is an open access publication, which can be used, distributed and reproduced in any medium according to the Creative Commons CC-BY 4.0 License.

Multimodal negative sentiment recognition of online public opinion on public health emergencies based on graph convolutional networks and ensemble learning

To improve the effect of multimodal negative sentiment recognition of online public opinion on public health emergencies, we constructed a novel multimodal fine-grained negative sentiment recognition model based on graph convolutional networks (GCN) and ensemble learning. This model comprises BERT and ViT-based multimodal feature representation, GCN-based feature fusion, multiple classifiers, and ensemble learning-based decision fusion. Firstly, the image-text data about COVID-19 is collected from Sina Weibo, and the text and image features are extracted through BERT and ViT, respectively. Secondly, the image-text fused features are generated through GCN in the constructed microblog graph. Finally, AdaBoost is trained to decide the final sentiments recognized by the best classifiers in image, text, and image-text fused features. The results show that the F1-score of this model is 84.13% in sentiment polarity recognition and 82.06% in fine-grained negative sentiment recognition, improved by 4.13% and 7.55% compared to the optimal recognition effect of image-text feature fusion, respectively. © 2023 Elsevier Ltd

Research on sentiment classification algorithm for Corona Virus Disease 2019 Weibo text

Since the outbreak of Corona Virus Disease 2019, it has had a significant impact on people's lives. In order to help the government grasp the social opinion and do more scientific and practical propaganda and public opinion guidance for prevention and control, and to fully reflect people's attitude toward the epidemic and provide data support for government departments to release epidemic prevention measures. This paper uses Corona Virus Disease 2019-related Weibo comments as the research object and analyzes their sentiment using deep learning algorithms. The number of characters in Weibo comments is usually less than 140, which belongs to the category of short texts. Due to the use of few words, random user language, and irregular grammar, these texts have poor performance in text separation and word vector expression, adversely affecting sentiment classification. In order to solve this problem, this paper constructs the BERT-DPCNN model for sentiment analysis of epidemic short texts, which can not only extract the sentence-level text dependencies but also effectively avoid the problem of gradient disappearance of deep neural networks. The experiments show that the BERT-DPCNN model has the best effect and is of great value for the sentiment classification of short epidemic text. © 2022 IEEE.